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ABSTRACT 



This research represented the systematic evaluation of 
the production and effects of the television series "Carrasco I endas, " 
a program designed to facilitate the bilingual education of Mexican- 
American children. Evaluation of the process of program development 
noted first that the series was completed and did reflect the in- 
tended instructional objectives. Suggestions for improvement of 
the process aspects included: (I) modification of the time schedule 

to al low for more feedback and revision of programs during the 
production schedule, (2) an improved definition of the responsib! I i— 
ties of supervisory staff members, and (3) a closer working relation- 
ship between the curriculum and production supervisors. 

Product evaluation, or assessment of the effects of the 
series, was conducted by means of a field experiment involving 
children from the target audience population and by a survey of 
schools which used 1,,3 program. Results of the field experiment 
indicated statistically significant learning gains of television 
viewers (as against a control group) in English tests of multicul- 
tural social environment, English language skills, physical environ- 
ment, and cognitive development. Viewers also gained significantly 
In measures of overall language performance in both Spanish and 
English. These results had generality across first and second grades 
(except multicultural knowledge which increased only for second graders) 
and across classrooms either engaged or not engaged in bi 1 ingual^it,— 
struct ion programs. 

The school surveys, although indicating a major use of 
the program, did reveal a possible shortcoming in that a significant 
number of schools, even in heavy populated Mexi can-Ameri can areas, 
had no knowledge of the program's availability. Teachers' altitudes 
were substantially positive concerning the overall program and its 
use in the classroom. Detailed responses of teachers provided a 
basis for suggesting program revisions. 
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SECTION I 



I ntroduct t on 



This is a report of the evaluation of the first year's 
production and use of "Ca rrasco I endas , " a thirty program television 
series designed to facilitate the bilingual instruction of Mexican- 
American children, primarily in grades one and two. Two tvpes of 
evaluation were undertaken. One, process evaluation, was an in- 
quiry into the development of the program. Three main questions 
guided this phase of the evaluations What were the major steps 
in the overall process of producing "Carrasco I endas?" What was 
the relationship between proposed instructional objectives and 
those realized in the program segments? Relative to job descrip- 
tions, what was the performance assessment of personnel in carrying 
out the res pons ibilities of the positions of televisi on coord i nator , 
curriculum and staff development coordinator, executive producer, 
advisory committee, and the evaluation team? 

The second type of evaluation, that on the produ ct, was 
an attempt to assess the effects of the program upon the target 
population of children, and upon the attitudes of the classroom 
teachers. The effects of the program upon the student population 
were assessed by the use of an experimental design which provided 
a basis for comparing learning gains of children who watched as a 
group against those who did not watch the program series. Further, 
it was possible to compare viewers and nonviewers relative to first 
and second graders, and relative to children who were in a class- 
room organized for bilingual instruction as against children who 
were not in such classrooms. Learning gains of the children were 
assessed relative to instructional objectives of the program series 
bearing upon symbolic representat ion (letters, numerals, geometric 
forms), computat i ona I ski I Is , cognitive processes . the phys i ca I 
envj ronmen t , multicultural social environme nt, and language skills. 
The specific objectives in these areas were divided between items 
pertinent to Spanish subculture and language as well as English 
subculture and language. The impact of the program upon teachers' 
attitudes was assessed by use of guesti onnai res and observation 
notebooks which provided information on the utilization of the 
television series throughout the viewing area, the detailed atti- 
tude of teachers toward the program and its use in the classroom, 
as we I I as recorded observations made on a program by program basis. 

Section 2 of this report describes the results of the 
process ©valuation; section 3 the product evaluation relative to 
the impact of the program series upon children's learning; sections 
4 and 5 report upon teachers' reactions to the program series. 

1 
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SECTION 2 



Process Evaluation 



In examining the development of a series of television 
programs, it Is often as important to review the processes which 
went into the dev&'opment of the programs as it i s to review the 
resulting programs and their effects. An examination of the 
processes which contributed to the final product provides Infor- 
mation concerning the participation of the various project com- 
ponents, the schedule, and the feedback loop which permitted self— 
corrective measures. A review of the process which resulted in 
this 30-program television series serves as input to future efforts, 
and it is in the spirit of providing f eed-forwa rd to the individuals 
charged with the second year’s development of "Carrasco I endas" that 
this process evaluation is carried out. 

2,1, What we re the major steps in the overal i process? 

In July, 1970, the curriculum staff at Region XIII began 
to develop the program In terms of content areas and behavioral 
objectives to be covered by the program series, A group of con- 
sultants met with the Curriculum Coordinator to review available 
materials and fo surface materials In areas related to the Mexican- 
Ameri can chi Id, Consultants in various content areas were brought 
in to assist the Television Coordinator and Curriculum Coordinator 
with the development of the programs. Program development was 
characterized by the Region XIII staff as follows: 

The content of the series is designed fo reinforce and 
supplement existing bilingual instructional programs. However, 
rather than duplicate a classroom situation, the television series 
takes advantage of the unique possibilities of the medium, employing 
a broad range of presentation techniques such as actors, puppets, 
films, and music, dance, and special effects to provide innovative, 
attractive and entertaining enrichment of classroom activities. 

The six content areas are being carefully sequenced by the Project 
Coordinate?’, Curriculum Coordinator, and Consultants In order to 
provide varied repetition of each area throughout the series. 

... Of special interest in the development of the series 
is the emphasis being placed on bilingualism, Mexi can-Ameri can 
culture, Mexican and Spanish heritage, and the multicultural society 
of the United States. Through, „ •+ the series the Spanish language; 
Mexi can-Ameri can actors; Mexi can-Ameri can personalities in cameo 
appearances; music, games, dances, rhymes, and other elements of 
Hispanic tradition; sets; participating children; etc. reflect this 
element of the series both in segments devoted specifically to this 
area and coordination with the other content areas. 
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Specifications for each program were submitted to the 
production staff at KLRN and to the evaluation staff at the Center 
for Communication Research. Meetings were held with all three 
aforement i oned components represented in order to agree upon a 
forma r for the curriculum staff’s spec i.f I cat I on of content and 
objectives which would be most effective as Input to both produc- 
tion and evaluation teams. 

The production staff at KLRN, upon receipt of the curri- 
cu I urn content and objectives for each program prepared a program 
script designed to carry out the curriculum staff ’ s directives. 

At the same time, the evaluation team prepared test I terns designed 
to measure the objectives in question. The script prepared by KLRN 
was then made available to the curriculum staff for critique and 
suggested revisions. The Advisory Committee was encouraged to offer 
suggestions concerning both content and objectives and their reali- 
zation in program scripts. After critique and suggested revisions 
were completes, the production staff began to videotape segments for 
the program. Some of the completed segments were used by the eval- 
uation team to investigate the interest level shown by children 
viewing these segments and attitudes of teachers and older children 
toward the segments. These evaluation efforts were designed to 
provide feedback to the program developers at an early stage in the 
process in order to permit modifications when Indicated by responses 
to the pre-vl ewed segments (cf, Fig, 2,1), 

2,1*1. Scheduling: The time line . The process described 

above suffered from time constraints. Curriculum development at the 
Education Service Center began later that originally planned (cf, 
2,3,1), The first curriculum content and objectives did not reach 
the production and evaluation teams until mid-September, This late 
start severely limited the overall feedback loop process, which, as 
described above, should have permitted a continuous series of check- 
points for possible revisions and program improvement. These time 
constraints caused, for example, little or no time for script critiques 
by the curriculum staff and Advisory Committee; critiques were often 
made after program segments had already been videotaped. Preview 
evaluations by the evaluation team were also of limited application 
in that feedback resulting from these previews could not be Incorpor- 
ated into future segments because the latter had themselves been 
videotaped. For example, many of the suggestions made by the Advisory 
Committee upon viewing the first finalized group of segments were of 
a very basic type (e . g . , the character i zati ons of some of the program’s 
main personages); these suggestions could not be implemented because 
of the large number of segments (in which these characters appeared) 
a I ready v i deotaped , 

2.1,2. Recommendat Ions . More time must be allotted be- 
tween the receipt by the production staff of the content and objectives 
of a given program and the actual videotaping of that program. Im- 
plicit here Is, of course, the need for early submission of objectives 
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to the production staff. This time is necessary + o permit all 
necessary feedback and critiques. Such feedback might take the 
form of pointing out discrepancies between content and objectives 
as envisioned by the curriculum staff and the program segments as 
created by the production staff. It might also include comments 
concerning linguistic usage in both Spanish and English, as well as 
other details involved in the presentation of curriculum cortent 
and objectives. The Advisory Committee and the evaluation team 
could also provide necessary feedback to the production and curri- 
culum staffs If sufficient lead time were made available. In short, 
if is absolutely imperative that the lead time between submission by 
the production staff of program scripts and acf ua I videotaping of the 
program segments be increased significantly. An experimental endeavor 
such as this requires maximizing possibilities for feedback from as 
many sources as possible. The mechanisms for such feedback already 
exist* what is needed is sufficient time to permit them to function 
effect! vely . 

2.2, What is the relationship between proposed ins t r uct iona I 
objectives and those realized in program segments? - 

Most of the instructional objectives stated by the curri- 
culum staff were i ncorporated into segments produced by KLRN, A 
complete tal ly of the objectives which were incorporated was made by 
KLRN's staff and appears in Appendix A. Close inspection of this 
information reveals that In many program segments several instruc- 
tional objectives were combined. 

Consultation with the curriculum staff revealed their 
satisfaction with the overall coverage provided by KLRN of the content 
areas and instructional objectives in the 30-program series. Most 
objections to segments raised by the curriculum staff involved details 
of linguistic usage or specific modes of presentation. 

2.2, I , Recommendat i ons . The very adequate coverage by 
KLRN of the content and Instructional objectives presented to them 
by the curriculum staff represents a positive factor in the overall 
process. On the other hand, many details which became points of 
contention between curriculum and production staffs could probably be 
more easily dealt with if more lead time were available for discussion 
and negotiation. It is obvious that controvers I a I aspects of a given 
program segment disturb all involved in the project; an opportunity 
to settle these issues before videotaping would contribute greatly to 
the program development process. This opportunity would be provided 
by insuring greater lead time and more frequent Interactions between 
curriculum and production staffs, 

2.3. Relative to job descriptions, what was the performance 
assessment in carrying out the responsibilities of the 
foliowing positions? 
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2.3, I, Television Coordinator . The duties and responsi- 
bilities of the Television Coordinator are described as follows (cf. 
Fig. 2.2): 

The bilingual television project coordinator, upon dele- 
gation by the director of program development, shall be responsible 
for continuing coordination and implementation of bi I ingua I tele- 
vision activities among the participating agencies, including Region 
XIII, ESC, KLRN-TV , schools, evaluators and the Advisory Board, as- 
suring that ail tasks are fully developed and kept on the time 
schedu I e. 



His specific ■functions with regard to Region XIII ESC are 
reporting to state and federal agencies as well as the Independent 
auditor as required; coordination of bilingual television activities 
with other programs of Region XIII ESC such as the Bilingual Project, 
Migrant Services, Media Services, Special Education, and Staff Develop- 
ment; and insuring that all activities of the project are compatible 
with Service Center Policies and federal guidelines. In addition, he 
cooperates with the Curriculum Coordinator to determine general and 
specific content and instructional objectives for the entire series. 
This involves the compilation of research findings in subject matter 
fields, educational television, educational psychology, and bilingual 
education for subsequent Input Into other activities of the project 
as well as the acquisition of consultant services as program needs 
and specific resources are identified. 

His specific functions with regard to KLRN-TV are providing 
liaison between the research and production elements of the project,' 
critiquing scripts, and attending taping and filming sessions in 
order to serve as the final authority regarding the meshing of program 
content and appropriate production in order to insure sound pedagogical 
practices. 



His connections with schools involve his coordination of all 
public relations activities including compilations of comprehensive 
mailing lists of educators within the KLRN-TV viewing area; publicity 
for the Invitational Workshops, staff development workshops, and the 
series itself; mailings of Teacher Manuals; and personal contacts 
through his availability for staff, development activities upon re- 
quest. 



He is further responsible for reporting to the Advisory 
Board and the evaluators as to overall management and progress of 
the project and relaying these groups* input into the appropriate 
project elements. 



One of the serious problems in the program development 
process was that the Television Coordinator did not join the project 
until approximately one month after its inception. This delay was 
problematic in two areas: (!) it resulted in his requiring time to 
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Figure 2.2. Organizational Chart of Education Service Center, ' Independent 

Region XIII with Bilingual Television Component. v Educational 
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catch up with activities conducted before his arrival, thus delaying 
the presentation by the curriculum staff of the program content and 
instructional objectives for the first programs in the series and 
reducing the overall lead time for program feedback opportunities; 
and (2) because the project was initiated without his active partici- 
pation, his role delineation did not seem to be clear for some months 
after joining the staff. These problems resulted in some confusion 
in the early months of program development. 

Another problem appeared to result from the geographical 
distance which separated the Television Coordinator and Curriculum 
Coordinator from the production staff. Working as two entirely 
separate entities most of the time, the curriculum staff and the 
production staff often suffered from poor communication, 

2. 3. I. I. Recommendations , The Television Coordinator 
should, from the start, be ful ly responsible for and involved in 
program deveiopment. This recommendation includes his working in 
close contact with the production staff on a daily basis. It seems 
imperative, based upon broken communication channels during the past 
year, that curriculum and production staffs be housed in the same 
general area so that last-minute questions and details can be easily 
discussed and settled before ind even during videotaping. The geo- 
graphical separation establishes an unnecessarily cumbersome barrier 
to smooth program development, 

2.3.2. Curriculum and Staff Development Coordinator . The 
role of the Curriculum and Staff Development Coordinator was defined 
as follows (cf. Fig. 2,2): 

The Coordinator for Curriculum and Staff Development, upon 
delegation by the project coordinator, shall have the major responsi- 
bility for developing objectives, content, and teacher guide materials 
for the bilingual television programs. 

Her specific responsibilities toward KLRN would be: 

. To provide all instructional objectives for the television 
program. 

, To advise KLRN where and when the Consultants will be 
giving their presentations in regard to the different 
di sci p I i nes . 

• To share the responsibility with project coordinator in 
the reading of scripts and viewing of segments to deter- 
mine whether or not objectives have been met. 

. To identify materia Is and resources and report them to 
KLRN. 
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in regard to the Teacher Guided, the Curriculum Coordinator will; 
Determine the areas of content. 

. Select the consultants for the different content areas. 

. Write all the instructional objectives. 

. Write supportive classroom activities to follow up the 
viewing of the programs. ( i hese would be an important 
part of the Teacher Guides.) 

Compose the Teacher Guide for the TV Program. 

. Make the necessary arrangements for the printing of the 
Teacher Gui des. 

Her connection with the schools will involve: 

. Visitations to classrooms for feedback purposes to see if 
materials and programs are suitable to the needs of the 
children in their area. 

• Helping teachers (perhaps in workshop situations) to de- 
velop materials from the Teacher Guides. 

. Providing the teachers with tapes of the songs, rhymes or 
stories from the programs thar are best suited for class- 
room use . 

T ^ e Curriculum Coordinator is also responsible for working 
with community agencies and parent groups in keeping them abreast of 
the program. 

There was some apparent confusion in the early months of 
the project concerning the exact specification of the role of the 
-u, ricu I urn Coordinator, Because the individual selected for the 
position had special talents in the area of educational materials, 
it was natural that she leaned toward making contributions along 
these lines. During the initial phase of the project, however, the 
greatest need was for complete specification of content areas and 
instructional objectives for the first programs in the series. 

^reject delays could probably have been avoided if the Curriculum 
Coordinator's role had been more carefully delineated from the start. 
In addition, because of the severely constrained time schedule, the 
Curriculum Coordinator's efforts at producing Teacher Guides for the 
50 programs were frustrated, and these guides were not as successful 
as they might have been (cf, 4,2,2). 

2.3.2. I, Reeommendat ions . The principal recommendation 
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involving the Curriculum Coordinator concerns the question of geo- 
graphical separation of the production and curriculum staffs dis- 
cussed in 2. 3.1. I above. It is also recommended that the Curriculum 
Coordinator's contributions to actual production be very carefully 
specified to prevent misunderstandings concerning delineation of 
responsibilities. The Curriculum Coordinator should also be provided 
adequate time for the development, printing and distribution of com- 
prehensive Teacher Guides. 

2.3.3, Executive Producer . The Executive Producer's role 
in overall program development is described as follows (cf. Fig. 2.3) 

Responsible for design and coordination of the production 
of 30 half-hour programs and 4 in-service programs. Responsible for 
supervision of all aspects of production, including films, animation, 
music, visuals, props, scenery, puppets, costumes, staff, scripting, 
and KLRN-ori g i nated publicity. 

Res pons ible for the hiring and di rect ion of all full -t I ne 
and part-time production personnel connected with "Carrasco I endas . " 

Responsible for reporting on all aspects of production to 
Region XI I I Education Service Center, the Advisory Board, the Com- 
munications Research evaluation team, and the Office of Education. 
Also represent KLRN-TV at all conferences, staff meetings, workshops, 
and conventions, ■ 

Responsible for coordination of KLRN efforts to expand and 
distribute the "Carrasco I endas" series. 

Act as host-i nstructor on the series and as narrator for 
numerous filmed and videotaped segments. Travel to filming on loca- 
tion, and appear in the first film of the series. 

Prior to the production of the series participate In class- 
room observation and consult various educational specialists. 

Owing to the time restrictions which characterized this 
project from the outset, many aspects of production were less ef- 
ficiently carried out than they might have been. Program producers 
indicated repeated I y that segments for several different programs 
utilizing the same sets, lighting, and actors should be most effic- 
iently taped on the same occasion. Because program scripts were 
written one by one, upon receipt of program content and objectives 
from the curriculum staff, this more efficient operation was not 
poss i b ie. In addition, It was noted that program producers were 
unusual ly sensitive to criticism of any kind regarding completed 
segments. 



2*3.3. I, Recommendat ions . Two recommendations are apparent 
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from first-year program production. The f i rst 'concerns closer 
coordination between curriculum and production staffs; this co-* 
ordination should permit a greater give and take regarding program 
development. Criticisms should, as a result, be viewed as coming 
from within a joint effort rather than as being leveled by out- 
siders, The second recommendation concerns the greater lead time 
between development of program scripts and final taping* This in- 
crease in lead time would permit not only adequate opportunity for 
settling curriculum and production differences, but also, in as- 
sembling the scripts for several programs, the product! staff 
should be able to tape segments in a more efficient and Inexpensive 
manner , 

2,3,4, Advisory Committee . The role of the Advisory 
Committee in overall program development was described as follows 
(of , Fig, 2,2) i 

The primary role of the members of the Advisory Committee 
Is to provide representat ? on of bilingual experts. Region XIII 
Education Service Center Staff, school district personnel, princi- 
pals, teachers, and parents in formulating, critiquing, and providing 
suggestions in the following areas: 

a. Appropriate consultants 

b. Pertinent research materials 

c. Program content 

d . Instructional objectives 

@* T.V, presentation techniques 

f. Staff development 

1) Invitational workshops 

2) In-service program 

3) Teacher study-gu i des 

4) Supportive classroom materials 

g. Program utilization 

h. Community involvement 

As it evolved, the role of the Advisory Committee was 
never clear. After a period of sporadic activity, the Advisory 
Committee discontinued its meetings. The first direct participa- 
tion by the Advisory Committee In program development occurred 
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only after many taping sessions had been completed and a few program 
segments were made available for thei - rovlew. The Committee pro- 
vided a sound critique of many aspects of the segments they viewed, 
but, as discussed more completely elsewhere (cf. 2 . 1 . 1 ), these 
criticisms were of a substantial nature, and could not have been 
incorporated because of the great financial investment already com- 
mitted to those program aspects being criticized. It was at this 
point that the evaluators suggested that a new role delineation for 
the Advisory Committee should be considered; if their input was to 
serve any constructive purpose, it would have to be received at an 
earlier point in program development. A proposal was made and ac- 
cepted that the Advisory Committee be divided into several sub- 
committees which would concern themselves specifically with one 
aspect of program development. Thus, there was to be a script 
critique s u b- comm i t tee, a production sub-committee, an evaluation 
sub-committee, and the like. After this specification of duties, 
the Advisory Committee completely ceased functioning. 

2.3.4. 1. Recommendat I ons . If program development is to 
expect the contributions of an Advisory Committee, careful arrange- 
ments must be made to delineate the exact areas in which contribu- 
tions may be made as well as the appropriato timing of such contri- 
butions. It is not beneficial for program developers to receive 
critiques after sol f-correcti ve measures are no longer possible; 
frustration on the part of the staff as well as the Advisory Commit- 
tee results. It must also be recognized that the Advisory Com- 
mittee is serving in a voluntary capacity; most Committee members’ 
schedules are fully committed to other activities, and there Is 
precious little time left for participating in television program 
development. It is not clear at this point just what the para- 
meters of Advisory Committee participation ought to be. The 
fact that the Advisory Committee ceased functioning once a spe- 
cific delineation of time-consuming participation was set forth 
would suggest that to expect more than occasional consultative 
participation Is unrealistic, 

2,3,3, Evaluation team's contribution to process . The 
evaluator's contribution to the program development process was in 
terms of formative evaluations. There were four such evaluations 
conducted by the evaluation team. Program segments prepar'd by KLRN 
were taped and previewed by different viewer groups to determine the 
groups’ reactions to those segments In the interests of providing 
feedback to both curriculum and production staffs concerning overall 
viewer preferences and segments having a high interest level. The 
first such evaluation was conducted at one of the elementary schools 
participating in the Region XIII bilingual program. Program segments 
were viewed by first and second graders. The children were observed 
during their viewing of the segments and were interviewed subsequent 
to viewing. Results of this preview appear in Appendix B. A second 
evaluation was conducted at a meeting of the Advisory Committee (cf. 
2,3,4). The results of this evaluation appear in Appendix C, The 
third preview evaluation was conducted with Mexiean-American Junior 
High School students. The results appear in Appendix D. The fourth 
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and last preview evaluation was conducted with a group of Head 
Start children in Austin, The purpose was to determine whether 
program segments were at all appropriate for preschool children 
as well as for Black and Anglo children. Ire results of this 
evaluation appear In Appendix E. All results from these preview 
evaluations were made aval I able to a I I components of program de- 
velopment immediately upon completion of data tabulation in order 
that ful I benefit be received from the feedback provided by the 
different preview audiences, 

2.3.5. 1 . Recommandat f ons . The evaluation team could 
play a far greater role In the formative evaluation of "Carras- 
CO I en das." It would be desirable that more feedback be obtained 
from both student and teacher populations during program development 
so that set f-corrective measures could be taken. In order to secure 
such formative evaluation data, and to permit substantial feedback 
from the evaluators, however, the overall lead time would have to 
be increased considerably. Many of the comments and suggestions 
which appear In Section 5 of this report, which were obtained only 
after the completion of the series, could have contributed greatly 
to program development had they been available during program de- 
velopment. Were there adequate time, such contributions could be 
collected and summarized by evaluators during program development, 
thus insuring meaningful feedback to program developers at a time 
when its impact would be greatest. 
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SECTION 3 



Product Evaluation: Test Data 



To determine whether watching Carrasco I endas resulted 
In childr@n f s attaining the objectives specified by the curri- 
cu I um staff, a field experiment was designed to assess program 
effects upon learning gains. 

3, I * Research Design 

In overview, the research involved the definition of 
experimental and control groups of Mexl can-Amer i can children who 
were distinguished by either watching the 30 program series 
(experimental group) or not watching it because of engagement in 
alternative activities (control group). The experimental and 
control groups were drawn from first and second grade classrooms 
and from classrooms either engaged In or not engaged In an ongoing 
bilingual classroom program, Pre- and post-treatment measures were 
of learning gains in the areas specified by the instructional ob- 
ject i ves of the program and In the use of both Spanish and English 
languages. Following the 30 program series. It was possible to 
determine learning gain effects that were due to viewing the pro- 
gram and what the generality of these effects was across first and 
second grades, and across bilingual and nonbi lingual classrooms. 

3,2. Subjects 

Subjects chosen for participation In this aspect of the 
product evaluation were Max! can-Amer i can first and second graders 
in a school which has a bilingual education program with both 
experimental and control classrooms. These children represented 
the target population of the series as specified In the project 
proposa I . 

A total of ■ 88 children were selected randomly from eight 
classrooms, four first grade and four second grade, half of which 
(two) on each grade level were experimental bilingual classrooms 
and half control classrooms. The breakdown of the sample appears 
in Figure 3.1. for the purposes of measuring the learning effects 
resulting from watching Carrascolendas, the group of 88 children 
was randomly divided into two groups, an experimental group of 
children who watched the television series, and a control group of 
children who were provided an alternative activity (cf. 3.5) during 
the broadcasts. As Is seen in Figure 3.1, the total number of 
children in the experimental (watchers) group was 48, and the total 
in the control (non-watchers) group was 40, It is worth noting 
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Exper i men+a I 
(TV watchers) 
. (n = 48) 





Bilingual 
(n - 10) 



Con+ro I 
(n = 10) 



Bill ngua I 
(n = 10) 



Con+ro I 
(n = 10) 



Figure 3,1, Subject Sample. 




Classroom A 
(n = 6) 
Classroom B 
(n = 6) 

Classroom C 
<n = 6) 
Classroom D 
(n = 6) 

Classroom E 
(n = 6) 
Classroom F 
(n = 6) 

Classroom G 
(n = 6) 
Classroom H 
(n * 6) 



Classroom A 
(n = 5) 
Classroom B 
(n = 5) 

Classroom C 
(n = 5) 

C I ass room D 
(n = 5 ) 

Classroom E 
(n = 5) 
Classroom F 
(n = 5 ) 

Classroom G 
(n = 5) 
Classroom H 
(n = 5) 
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that 88 children were selected to participate in the evaluation 
in order that two distinct criteria could be met: (I) a sufficient 

number of Ss to permit the use of certain statistical analyses of 
data, and (2) a small enough sample to permit in-depth interviews 
with each child in both English and Spanish. 

3,3, The Interview Instruments 

After exploring many test instrumanrs for the measurement 
of English and Spanish language skills, self-concept, cultural 
attitudes, and achievement in several content areas, it was decided 
that the most p roduct i ye means of evaluating possible I ea rn I ng ef- 
fects of the television series itself was to design a criterion- 
referenced test i nstrument based on the program content and objec- 
tives as set forth by the curriculum staff at the Education Service 
Center. The principal content areas to be dealt with In the tele- 
vision series developed by the curriculum staff (cf. 2.1) included 
the f o I I ow I ng : 

Symbolic Representation: Letters, Numerals, Geometric 

Forms 

Computational Ski I Is 

Cognitive Processes 

The Physical Environment: The Child and the Physical 

World Around Him 

The Multicultural Social Environment of the Mexican 

Amer I can Child 

Language Skills (Spanish and English) 

A breakdown of these major content areas appears in Appendix F. As 
discussed in 2.1, the curriculum staff prepared detailed descrip- 
tions of content areas and specific objectives to be carried out 
in each of the 30 programs making up the series. These content 
and objective guidelines for each program were provided the pro- 
duction staff as we I I as the evaluation team. Upon receipt of 
these objectives, the production staff determined the feasibility 
and most appropriate means of presenting them on the television 
program in question, and the evaluation team determined the feas- 
ibility of incorporating them into the assessment instrument. Many 
of the objectives were not included in the assessment Instrument 
because they required special equipment or techniques not easi ly 
achieved in the interview setting. Examples of such an objective 
are : 



8h. The child can correctly use the expression "to get 
embarrassed" In retelling a story in which the key 
point is a character's getting embarrassed. 
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I5a* Given sounds that are particularly associated 

with the country and "arm, the child can identify 
them as belonging to the farm and its environs. 

Some objectives were combined with others covering the same or a 
similar content area in order to, limit the length of the test 
Instrument* Where possible, several objectives were incorporated 
into a single test question. Two separate test instruments were 
developed according to the procedures just describee, one in 
Spanish and one in English, Some test items were identical (ex- 
cept for the language used) in both instruments; most were exclu- 
sive to one of the two instruments. 

Because of the research design which cal led for pre- 
and post-testing of all children* it was necessary that the test 
instruments be developed by early December with pre-testing to 
begin in early January, Objectives for only twenty of the total 
thirty television programs were available at the time the test 
instrument was being finalized. As a result, the test instruments 
include questions covering programs I through 20 and those segments 
in programs 20 through 30 which were repetitions of segments in 
the first 20 programs, 

3,3*1, Piloting of test instruments . The test Instru- 
ments completed in early December were subjected to pilot testing 
in order to determine the approp r I ateness of questions Included 
in the instruments. This pilot testing was conducted with children 
meeting the target population specifications in other schools in 
the area. As a result of the pilot testing* several revisions were 
made In the test questions as well as in the order of presentation 
of these questions. This pilot testing also provided the Inter- 
viewers with considerable practice In administering the test in- 
strument before pre-testing began* 

3.4. Test Administration 

The pilot testing of the test instruments was conducted 
by the two test adm i n i strators selected to conduct p re- and post- 
testing of the Ca rrasco I endas ©valuation. Both test adm i n i st rators 
were se I acted several months In advance of pilot testing. In order 
to ensure that the children parti ci pat i ng in the evaluation would 
feel as comfortable as possible during the interviews, the two 
interviewers spent every Friday morning during the months of 
September, October, November and December at the target school. 

The interviewers worked with the teachers In each of the eight 
classrooms selected for participation In the evaluation, and, as 
a result, became well known to the children who were to be tested. 
The importance of establishing the familiarity of the interviewers 
was considered a crucial factor in obtaining the best possible 
performance from the children during both pre- and post-testing. 
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Teachers and Interviewers were unanimous In their estimation of 
the success of the four-month effort at establishing rapport wivh 
the ch 1 I dren . 

One of the interviewers conducted a I I Engl I sh— I anguage 
interviews and the other ail Span I sh- I anguage interviews. Several 
criteria were used to select the i nterv i ewers . The English- 
language Interviewer was selected for her previous experience with 
and interest in working with young children as well as her very 
clear speaking voice and dialect which approximated Standard 
American Engl ish. The Span i sh- I anguage interviewer was selected 
on the basis of his previous experience and i nterest in working 
with sma I I ch I Id re n as well as his being a native spea ker of 
Spanish from the same dialectal region as the children to be tested, 

5 . 4 . 1 . Re I I ab ? I i ty of j nterv j ewe rs . Du ring the pi! ot 
tesfing of the test instruments, interviewers checked on the relia- 
bility of their interviews by interviewing the same child twice 
within a three-week period. No differences were found in the re- 
sponses of the children to the two separate interviews. No relia- 
bility between interviewers was necessary since all Interviews In 
each language were conducted by the same individual interviewer. 

3.4.2. Final versions of the test instruments . After 
pilot testing and analysis of chi I dren T s responses to that testing 
were completed, the English and Spanish test instruments were 
final i zed , Sample questions from the two test instruments appear 
in Appendix G. It wi II be noted that many of the test questions 
in both instruments cal I for the presentation of visual stimulus 
material (e.g.. Question 83 in the Spanish instrument). All such 
visual stimuli were prepared specifically for these test items. 

3-4.3. I nterv I ew p rocedu re^ The interviews were conducted 
in a large recreation room In a community center located adjacent to 
the school. Two tables about 12 yards apart, and an ample supply of 
chairs were available. Conducting the English and Spanish interviews 
In the same room, at the same time* and at not too great a distance 
apart, did not present many difficulties. The chief difficulty was 
a problem of distractions to the child. This problem was resolved 
by seating the child in a position which did not allow him to be 
distracted by the other child being Interviewed, by a television set 
which was in a corner of the room (The set was usually turned off or 
the volume was ser very low,), or by other people who occasionally 
entered the room. When it was necessary for both interviewers to 
share a table, cons i derat ion was given to providing as little dis- 
traction as possible to the child, usually by having appropriate 
seating arrangements. The volume in which both interviews were 
conducted was adjusted according to the distance which separated 
the interviews. The tape recorders were able to pick up very quiet 
conversat ! ons held close to the microphone. 
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The persons using the community cent er either avoided 
using the area In which the interviews were being conducted, or 
were very quiet If they did use part of the room. Occasional !y 
other smaller rooms or a kitchen were used for interviews when 
the main recreation room was occupied for meetings or other center 
act i viti es* 

The length of the interviews varied for each child. 
Generally speaking, first grade children took from 30 to 60 
minutes, with 35 to 40 minutes being the average. The second 
grade children averaged 30 minutes, and the usual range was from 
20 to 50 minutes. These time ranges were approximately the same 
for both English and Spanish interviews, and during pre- and post- 
test i ng , 

Each child was interviewed separately in English and in 
Spanish, Due to children’s fatigue from regular school activities, 
as well as from the interviews, each child was interviewed only 
once during a day* The English and Spanish interviewers decided 
together at the beginning of each day which children they would 
interview in order to avoid testing the same child twice In the 
same day , 

During pre-testing it was noted that first grade child- 
ren were tired and distracted any time after the noon hour* Ac- 
cordingly, the interviewers restricted first grade interviews to 
the morning hours when the children were more alert. Second 
graders did not show signs of fatigue until about 2 o’clock; thus, 
they were interviewed either in the morning or in the afternoon* 
Whenever a particular child showed fatigue, breaks, such as getting 
a drink of water or doing exercises, were incorporated Into the 
interview. If these did not prove sufficient to envlgorate the 
child, then the interview was discontinued until the following day. 
When the interview was resumed on a subsequent day, it was begun 
where the previous day’s Interview had ended* 

The Interviewer asked the teacher of the selected class- 
room to choose a child from the list of names of children to be 
Interviewed, Thus, children were selected randomly by the eva I - 
uators, and classroom teachers determined the order in which they 
were interviewed* This procedure was f o I lowed to accommodate a 
chi Id who might have been occupied In a task best left undisturbed, 
as well as to protect the teacher’s classroom schedule. Teachers 
were very cooperative In releasing children from the classroom; 
their attitude toward the research project as a whole was always 
excellent* After a child was selected, the interviewer and child 
walked to the testing site, engaging in light conversation in the 
language in which the Interview was to be conducted in order to 
establish rapport between interviewer and child. 

After entering the interview room, the child was seated* 
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During the pre-testing the child was seated on the right of the 
Interviewer to accommodate turning the pages of the book containing 
visual stimuli* During post-testing the child was seated on the 
left s • de of the interviewer; the interviewer was right-handed and 
needed this hand to score the child’s response on the interview 
score sheet. The pre-test; ng was tape-recorded and responses were 
scored by listening to the tapes after the interviews were com- 
pleted. In post-testing the procedure of simultaneously inter- 
viewing and scoring the child’s responses was found by the Inter- 
viewer to be feasible and this procedure was followed. Post-test 
interviews were also tape-recorded to permit the retention of a 
permanent record. 

Before beginning pre-test interviews, the interviewer 
talked to the child about the tape recorder and microphone, and 
showed him how to thread the tap© reel in order to reduce any 
anxiety or uncertainty the child may have had about tape recorders. 
The child was then instructed to speak In a loud and clear voice, 
to answer In the language In which the interview was being conducted, 
and to Indicate that he did not know an answer by saying ” I don’t 
know 1 ' rather than remaining silent. The ch F I d was also told that if 
he could not answer a question in the language of the interview, but 
knew the answer in another language (could say it ’’another way”), 
this answer would be acceptable. 

During post-test interviews, the recording equipment was 
only briefly mentioned by the interviewer. Because of the children’s 
familiarity with the interview environment, no extensive instructions 
preceded post-test Interviews. 

The interview consisted of questions which required verbal, 
non-verbal or manipulative responses. A notebook containing visual 
stimuli was used during the interview, as were a small wooden box, 
stones of different sizes and textures, a tissue, and laminated cards 
of different letters and numbers. The interviewer posed a question 
to a child and waited five seconds before repeating the question 
once. The Interviewer did not indicate to the child whether his re- 
sponse was correct or incorrect, but did use generalized, spaced 
re i nf orcements such as ’’Good” or ’’You’re doing fine” to encourage 
the child throughout the interview. 

If a child seemed nervous or reticent, the interviewer might 
place his hand on the chi Id’s shoulder to reassure him or make him 
feel more comfortable. This technique was usually effective. If a 
chi id was especial ly shy, the Interview was interrupted for conversa- 
tion or a trip to the water fountain. If an Interview seemed too 
difficult for a child because of his physical or emotional condition, 
it was discontinued until a subsequent day. In the few instances 
where postponement occurred, the interruption seemed to have a posi- 
t i ve result in that the chi Id resumed the interview enthus I ast I ca I ly. 
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At the end of the interview the Interviewer told the 
child that he had done a fine job and deserved a prize. A two- 
part prize was given each child: during pre-tasting, crayons and 

a coloring book were given by the English and Spanish i nterv i ewers , 
respectively- pencils and a pad of lined paper were prizes during 
post-testing. Each first and second grade classroom also received 
a set of wooden letters or numbers for the general class use. All 
the chi Idren were eager to be interviewed, and since only a select 
number were chosen (by random sample), the remaining children were 
acknowledged with the classroom prize. 

After each interview the interviewer accompanied the child 
back to his classroom and returned with the next child to he inter- 
V I ewed . 

All pre-testing was completed on both experimental 
(watchers) and control (non-watchers ) chi Idren before the broadcast 
of the f i rst program in the series on February 15, 1971, 

3 # 5 , Alternative Activity for the Control (Non-watcher) Group 

In order to carry out the experimental design proposed 
for the Car rasco I endas evaluation, it was necessary, as indicated 
earlier, that a specified group of children serve as a control or 
non-watcher group. It was equal ly important, however, that the 
chi Idren who were so designated in no way fee) that they were being 
deprived of a privilege enjoyed by their television-watching class- 
mates, . Accord! ng ly, an alternative activity was planned and carried 
this act i v i ty was to be as att ract i ve to ch i I d ren as rema i n i ng 
in the classroom to watch the television programs, and, at the same 
1 1 me , to be ©ducat I on a I in natu re • Save ra I possibi I [ties we re ex— 
piored with the teachers of those children involved in this re- 
search project, and it was decided that the alternative activity 
would be a photography project in which the control children would 
meet three times per week during the television broadcasts to do a 
photographic essay of their school and neighborhood. With the 
cooperation of several of the teachers and the teacher aides, one 
* member of the evaluation team (whose own area of special ization 
was photography) conducted these photography sessions. Cameras 
and film were generously provided by the Education Service Center’s 
Bi I ingual Component, 

Each broadcast day, the control children assembled with 
the teachers, teacher aides and the evaluation team member to ex- 
plore with cameras their school and neighborhood. The 40 children 
were organized into five film crews who competed for taking the 
best pictures. The children were absent from their classrooms for 
approximate! y one hour every Monday, Wednesday and Friday morning, 
leaving their classrooms approximately fifteen minutes before the* 
broadcast (to permit teachers to engage in motivational activities 
for the television program) and continuing their photographic 
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